Exploring Temperature Sensitivity in LLM Outputs

Welcome back! In the previous lesson, you learned how to measure and interpret token usage in large language models (LLMs). That knowledge is important for understanding how efficiently a model processes information. In this lesson, we will focus on a different aspect of LLM behavior: the temperature parameter.

The temperature setting is a key way to control how creative or predictable a model’s responses are. When you send a prompt to an LLM, the model can generate many possible outputs. The temperature parameter lets you adjust how much randomness the model uses when picking its next word or phrase. A low temperature makes the model more focused and deterministic, while a higher temperature encourages more variety and creativity in the responses.

Understanding temperature is important for benchmarking because it helps you see how the same model can behave differently depending on this setting. By the end of this lesson, you will know how to run simple experiments to observe these differences and interpret what they mean for your use case.

The temperature parameter is a number, usually between 0 and 2, that you can set when generating text with an LLM. When the temperature is set to 0, the model will always pick the most likely next word, making its responses very predictable and consistent. As you increase the temperature, the model becomes more willing to take risks and choose less likely words, which can make its responses more creative or surprising.

If you set the temperature above 1, the model becomes even more random and unpredictable. While the model is still valid and will generate output, the responses may start to lose coherence or relevance, as the model is more likely to select unusual or unexpected words. This can be useful for brainstorming or creative writing, but may not be suitable if you need reliable or factual answers.

On the other hand, setting the temperature to a negative value is not valid. Most LLM APIs will return an error or ignore the setting if you try to use a negative temperature. Always use a value of 0 or higher to ensure the model behaves as expected.

A note on models and providers: the models used in this example are set up for you on CodeSignal. If you run similar code on your own machine, check your provider's documentation first: model names, API keys and supported parameters (such as temperature) differ between providers and models.

Let’s look at a practical example to see how temperature affects model outputs. In this example, you will use the OpenAI Python client to send the same prompt to the model three times, each with a different temperature setting. The prompt asks the model to describe a completely fictional animal found in a magical forest.

Here is the code you will use:

from openai import OpenAI

client = OpenAI()

prompt = "Describe a completely fictional animal found in a magical forest."

temperatures = [0.0, 0.7, 1.2]

for temp in temperatures:
    response = client.chat.completions.create(
        model="deepseek-ai/DeepSeek-V3",
        messages=[{"role": "user", "content": prompt}],
        temperature=temp,
        max_tokens=150
    )
    reply = response.choices[0].message.content.strip()
    print(f"\nTemperature: {temp}")
    print(f"Response: {reply}\n")

In this code, you first define your prompt and a list of three temperature values: 0.0, 0.7, and 1.2. For each temperature, you send the prompt to the model and print out the response. The only thing that changes between each run is the temperature value.

When you run this code, you might see output like the following (your results may vary):

Temperature: 0.0
Response: The Glimmerfox is a small, agile creature with shimmering silver fur and bright blue eyes. It is known for its ability to blend into the moonlit mist of the magical forest, making it nearly invisible to predators and travelers alike.

Temperature: 0.7
Response: The Glimmerfox is a rare animal with iridescent fur that changes color depending on its mood. It has long, feathery ears and a tail that glows softly in the dark. The Glimmerfox is said to bring good luck to anyone who spots it in the magical forest.

Temperature: 1.2
Response: Deep in the magical forest lives the Whimsyhorn, a floating, jelly-like creature with rainbow stripes and a single spiraled antler. It sings lullabies to the trees at night and leaves trails of sparkling dust wherever it drifts.

Notice how the response at temperature 0.0 is very straightforward and safe, while the response at 0.7 adds more creative details. At 1.2, the model invents a completely new animal with imaginative features. This shows how increasing the temperature leads to more diverse and creative outputs.

In this lesson, you learned what the temperature parameter is and how it affects the behavior of large language models. You saw that low temperature values make the model’s responses more predictable, while higher temperature values encourage creativity and variety. You also worked through a code example that compared model outputs at different temperature settings, helping you see these effects in action.

Next, you will get a chance to practice running your own temperature experiments. Try different prompts and temperature values to see how the model’s responses change. This hands-on practice will help you build intuition for when to use different temperature settings in your own projects.